Predicting Summary Quality using Limited Human Input
نویسندگان
چکیده
We present four experiments with summary evaluation approaches that use little or no human input in the form of model summaries or human judgements. We investigate whether system-produced summaries can be used to improve predictions of summary quality when few or no human summaries are available. We also validate our previous findings that measures of input-summary similarity and input cohesiveness are predictive of summary quality. We analyze the performance of our methods in predicting the human assigned scores for summarization systems from the 2008 and 2009 Text Analysis Conferences. Input-summary similarity metrics obtain correlations of about 0.7 with manual pyramid scores on the TAC ’09 data. Using only a collection of system summaries in place of gold standard, the correlation is 0.9. We also show that properties of input cohesiveness can predict the average system score with good accuracies.
منابع مشابه
A survey on Automatic Text Summarization
Text summarization endeavors to produce a summary version of a text, while maintaining the original ideas. The textual content on the web, in particular, is growing at an exponential rate. The ability to decipher through such massive amount of data, in order to extract the useful information, is a major undertaking and requires an automatic mechanism to aid with the extant repository of informa...
متن کاملPrediction of Egg Production Using Artificial Neural Network
Artificial neural networks (ANN) have shown to be a powerful tool for system modeling in a wide range of applications. The focus of this study is on neural network applications to data analysis in egg production. An ANN model with two hidden layers, trained with a back propagation algorithm, successfully learned the relationship between the input (age of hen) and output (egg production) variabl...
متن کاملDetermining water quality along the river with using evolutionary artificial neural networks (Case Study, Karoon River , Shahid Abbaspur-Arab Asad reach)
Rivers are important as the main source of supply for drinking, agriculture and industry.However, drinking water quality in terms of qualitative parameters, is the most important variable. Studias and predicting changes in quality parameters along a river, are one of the goals of water resources planners and managers. In this regard, many water quality models in order to maintain better water ...
متن کاملCar paint thickness control using artificial neural network and regression method
Struggling in world's competitive markets, industries are attempting to upgrade their technologies aiming at improving the quality and minimizing the waste and cutting the price. Industry tries to develop their technology in order to improve quality via proactive quality control. This paper studies the possible paint quality in order to reduce the defects through neural network techniques in au...
متن کاملPredicting the Air Quality Index of Industrial Areas in an Industrialized City in India Using Adopting Markov Chain Model
Introduction: The rapid urbanization coupled with industrial development in Indian cities has led to air pollution that causes adverse effects on the health of human beings. So, it is crucial to track the quality of air in industrial areas of a city to insulate the public from harmful air pollutants. The present study examined and predicted air quality index levels in industrial areas located ...
متن کامل